NHK STRL at TRECVID 2011: Surveillance Event Detection and Semantic Indexing

Laboratories participated in two tasks at TRECVID 2011: a surveillance event detection task and a semantic indexing task. Surveillance event detection is the act of detecting specific human actions from surveillance videos of crowded areas such as airports. This year, we targeted " Pointing, " " CellToEar, " " ObjectPut, " and " Embrace " actions, which our proposed system identified by using the bag-of-features model. We used a motion-appearance histogram that was calculated from a key-point trajectory as a feature. The trajectories were segmented on the basis of their detected position, and each segment functioned as a bag for the bag-of-trajectory approach. Evaluation results showed that our system has a relatively robust performance when detecting small human actions. Semantic indexing is the task of detecting semantic concepts (such as objects or events) from a large video archive. The common method for semantic indexing is the " bag-of-visual-words " approach [1, 2] which is based on a frequency histogram of local features including SIFT [3] or SURF [4]. The effectiveness of this approach has been shown by many previous works [5]; however, the method cannot take into account the relationship between feature points. It also suffers from a loss of information when converting the feature descriptor to a visual word. This paper proposes a novel local feature method that considers the spatial relationship between feature points and the co-occurrence frequency of feature descriptors at each feature point. The proposed method uses random forests algorithm [6] as a classification method to reduce the computational time required for training and detection. To evaluate its effectiveness, the proposed method is applied to a full task of 346 concepts. The demand for technology that can automatically identify human actions is increasing with the rapid spread of surveillance cameras. In addition to the use of surveillance, human action recognition techniques can be applied to many services, such as motion-based video searches and man-machine interfaces. In those applications, a person's full body shape can only rarely be captured from the input cameras. Therefore we targeted relatively small events that involve only part of a person's body and not motion of the entire body, such as " Pointing, " " CellToEar, " " ObjectPut, " and " Embrace. " Key-point trajectories around a human body contain rich temporal information on the person's motion[7, 8], so we used key-point trajectories as a feature for detecting events. Our system …